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Abstract 

Among various possible routes to extend entropy and thermodynamics to 
nonequilibrium steady states (NESS), we take the one which is guided by op- 
erational thermodynamics and the Clausius relation. In our previous study, we 
derived the extended Clausius relation for NESS, where the heat in the origi- 
nal relation is replaced by its "renormalized" counterpart called the excess heat, 
and the Gibbs-Shannon expression for the entropy by a new symmetrized Gibbs- 
Shannon-like expression. Here we concentrate on Markov processes describing 
heat conducting systems, and develop a new method for deriving thermodynamic 
relations. We first present a new simpler derivation of the extended Clausius rela- 
tion, and clarify its close relation with the linear response theory. We then derive a 
new improved extended Clausius relation with a "nonlinear nonequilibrium" con- 
tribution which is written as a correlation between work and heat. We argue that 
the "nonlinear nonequilibrium" contribution is unavoidable, and is determined 
uniquely once we accept the (very natural) definition of the excess heat. Moreover 
it turns out that to operationally determine the difference in the nonequilibrium 
entropy to the second order in the temperature difference, one may only use the 
previous Clausius relation without a nonlinear term or must use the new relation, 
depending on the operation (i.e., the path in the parameter space). This peculiar 
"twist" may be a clue to a better understanding of thermodynamics and statistical 
mechanics of NESS. 
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1 Introduction 

1.1 Background and motivation 

To develop a universal statistical mechanics for nonequilibrium steady states (NESS) is 
a major remaining challenge in theoretical physics. It has been expected since the early 
days in the research that to pin down a physically natural entropy (or free energy) can 
be a crucial step in such a project. It should be realized, however, that the entropy 
in equilibrium systems plays several essentially different roles; it is a thermodynamic 
function whose derivatives correspond to physically observable quantities, it is a quan- 
titative measure of which adiabatic process is possible and which is not, it is also a 
large deviation functional governing the fluctuation of physical quantities. There is a 
possibility that this "degeneracy" is an accident observed only in equilibrium states, 
and the degeneracy is immediately lifted when one goes out of equilibrium. If this is 
the case we shall encounter more than one "nonequilibrium entropies" each of which 
characterizing different physical aspect of a NESS. It will then be important to clarify 
what physics of NESS is represented by which extension of entropy. Among various 
promising attempts, we refer to the phenomenological consideration in [lj which tried 
to extend the notion of adiabatic accessibility, and recent explicit characterization of 
the large deviation functional in [21 El S] ■ 

In our own attempt to extend the notion of entropy to NESS [5j, we have concentrated 
on the aspect which gave birth to the concept of entropy, namely, the Clausius relation in 
thermodynamics. We started from a microscopic description of a heat conducting NESS, 
and showed that a very natural generalization of the Clausius relation, in which the heat 
is replaced with the "excess heat" , is indeed possible when the "order of nonequilibrium" 
e is sufficiently small. This was a realization of the early phenomenological discussions in 
[6l[T], and extension of the similar result in [7] for models with Gaussian thermostat. We 
also found that the microscopic representation of the entropy differs from the traditional 
Gibbs-Shannon form, and requires further symmetrization with respect to time-reversal 
transformation (see (I2.15P below). 
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In the present paper, where we treat a class of Markov processes describing heat 
conducting systems, we shall go one step further to show a new thermodynamic rela- 
tion that has the same entropy as in [5], but contains an extra correction term which 
is intrinsically "nonlinear nonequilibrium" and has no counterpart in the original Clau- 
sius relation. Although one might worry that showing new relations with new nonlinear 
terms is a rather arbitrary uncontrolled attempt, we argue that this extension is manda- 
tory and unique if we regard the extended Clausius relation of [5 J as a starting point. 
Moreover we shall see that to determine the entropy to the second order in the "order 
of nonequilibrium" e, one may only use the naive extended Clausius relation f)2.14p or 
must use the new relation (I2.17P with an extra term, depending on the operation un- 
der consideration. This fact suggests that there is a delicate but unavoidable "twist" in 
operational thermodynamics for NESS. We believe that one must understand the true 
nature of this twist before developing a satisfactory statistical theory of NESS. 

1.2 Brief summary of the setting and the results 

Before discussing the details, let us present a brief and informal summary of our approach 
to NESS and major results in the present paper. We shall in particular explain the 
important notions of excess heat and excess entropy production (in the baths). 

Let us focus on the simplest nontrivial example, i.e., a system (of, say, a fluid) 
attached to two heat baths with inverse temperatures f}\ and /3 2 . We assume that the 
system is characterized by a parameter (or a set of parameters) v that can be controlled 
by an outside agent. A typical example of v is the volume of the system. See the 
left-most figure in Fig. [2J 

Stepwise operation in equilibrium: We start with the case where the inverse tem- 
peratures of the two bath are both equal to /3. Suppose that we fix both (3 and v. Then 
after a sufficiently long time, the system reaches a unique equilibrium state characterized 
by (3 and v. 

We consider a thermodynamic operation to the equilibrium state. Suppose that an 
external agent instantaneously changes the inverse temperature (of both the baths) from 
j3 to P' and the parameter from v to v' . The system is no longer in equilibrium after this 
sudden operation, but finally relaxes into the new equilibrium state corresponding to /3' 
and v' after a sufficiently long time. Let us denote by Q the total heat that flows into 
the system from the heat baths during the relaxation process, and by (Q} its average 
over possible paths (i.e., histories). See Fig. [TJ Then the celebrated Clausius relation 
states that 

SeqO?'; v') ~ S cq W; u)=0 (Q) + 0(<5 2 ), (1.1) 

where S eq ((3; v) is the entropy of the equilibrium state, and S is the dimensionless measure 
of the changes 0'—/3 and v' — v. The error term 0(S 2 ) simply reflects the stepwise nature 
of the operation, and is not essential. 
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(Q) 

Figure 1: Thermodynamic operation in an equilibrium state. Left: One starts 
from the equilibrium state with the inverse temperature ft and the parameter v. 
Middle: At a certain moment an external agent abruptly changes the inverse tem- 
perature of the baths and the parameter. The system is no longer in equilibrium. 
Right: After a sufficiently long time, the system relaxes to the new equilibrium 
corresponding to ft' and v' . The heat (Q\ absorbed by the system during the 
relaxation plays an essential role in the Clausius relation (jl.ip . 



Stepwise operation in NESS: Our goal is to extend the previous consideration 
for equilibrium states to NESS. Let fti ^ P2, and fix fti, fti and v. Then again after a 
sufficiently long time, the system is expected to settle to a unique which exhibits 

no macroscopic changes but has a nonvanishing heat current. 

We consider a thermodynamic operation similar to the equilibrium case. Suppose 
that the system is in the NESS characterized by fti, Pi, and v. Then the external agent 
instantaneously changes the inverse temperatures of the baths from fa to ft[ and fa to 
j3' 2 , respectively, and the parameter from v to v 1 . The system is no longer in a steady 
state after the change, but will eventually relax to a new NESS characterized by ft[, fa, 
and v' . We want to examine if there is a relation like the Clausius relation (11.11) which 
holds in this situation. 

Let us now see that a naive extension of (11. ip is just impossible. Recall that, in the 
equilibrium case, the quantity —ft (Q) is precisely the increase of entropy in the baths 
during the relaxation process. Thus the Clausius relation (II. ip simply states that the 
change in the total entropy of the system and the baths is at most of 0(5 2 ). In a NESS, 
on the contrary, there is a constant increase in the entropy of the baths even when one 
makes no operation. To see this consider a NESS with fa, fa, and v, and suppose that 
fti < Pi- Then there is a constant heat current J| s > from the first heat bath to 
the system and the same amount from the system to the second bath. One then easily 
finds that the total entropy in the baths grows linearly in time with a constant rate 
[fti — Pi) Jf s > 0. Thus, in the operation to NESS, a naive counterpart of the quantity 
—ft (Q) is growing constantly before the operation and keeps on growing constantly 
after the operation. There is no hope that a thermodynamic relation like (11.11) holds. 

Excess heat, excess entropy production: To overcome this difficulty of divergence, 
we shall consider a properly "renormalized" version of the diverging "bare" quantity. In 



5 We are assuming that the temperature difference |/3i — /^l is sufficiently small. 




Figure 2: Thermodynamic operation in a NESS. Left: One starts from the NESS 
with Pi, /?2, and v. Middle: At a certain moment an external agent abruptly 
changes the inverse temperatures of the baths and the parameter. The system 
is no longer in a steady state. Right: After a sufficiently long time, the system 
relaxes to a new NESS. In the extended Clausius relation (jl.3p . the excess heat 
or the "renormalized" heat {{Qf 1 ), (Qf*)) plays an essential role since the "bare" 
heat diverges in time, (see also Figj3j) 




Figure 3: (Ji(i)) denotes the averaged heat current from the first bath to the 
system at time t. The area of the region below the solid line corresponds to the 
total heat from the bath to the system, which diverges in time. If one subtracts the 
heat necessary to maintain the NESS, one gets the shaded area which corresponds 
to the excess heat (Qi x ). This is a very natural strategy to "renormalize" diverging 
"bare" heat to get a finite quantity. 
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the present situation a natural procedure of the renormalization is to subtract "house- 
keeping heat" from the diverging "bare heat" in the spirit of Oono and Paniconi [Tj. 

To be precise let </&(£) (k = 1, 2) be the heat current that flows from the k-th bath to 
the system at time t, and (Jk(t)) be its average. The total heat Qk from the k-th. bath 
is given by the integral J dt Jk(t), whose average (Qk) clearly diverges as the range of 
the integral is enlarged. 

Let J| s (/3i, /3 2 ; v) be the heat current from the fc-th bath to the system^] expected in 
the NESS characterized by fa fa and v. Suppose that we make the sudden change 
of the parameters at time t — 0. Then the current needed to maintain the NESS is 
^(fafa v) for t < 0, and Jf{fafa v') for t > 0. We define the excess heat (or the 
renormalized heat) by substituting these steady heat currents (which correspond to the 
"house keeping heat" in [Tj) from the actual current (J^if)). Thus the excess current 
is given by (j k (t)) - J* k \fa fa u) for t < 0, and (J k (t)) - Jf{fafau') for t > 0. 
The excess (or renormalized) heat Q^ x , which is expected to characterize intrinsic heat 
transfer caused by the operation, is obtained by integrating this excess current. By 
noting that (Jk{t)) = ^| s (/?i, fa v) for t < 0, we have 

(qt) J™ dt {(j k (t)) - w^y)}, (i.2) 

which, unlike the total heat, is expected to be finite. See Fig. |3j 

Extended Clausius relation and its improvement: Our finding in [5] is that there 
is a sensible extension of the Clausius relation (11. ip in which the heat is replaced by the 
excess heat. More precisely we showed the extended Clausius relation 

S(/3[, v') - S(p x , fa v) = (Qf ) + /3 2 (QT) + 0(e 2 5) + 0(5 2 ), (1.3) 

where the nonequilibrium entropy S(0i,02] v) is a we h defined function of (3i, (3 2 , and 
z/, and coincides with the equilibrium entropy if $\ = (3 2 . The quantity — {flx (Qf") + 
(3 2 (Q 2 X )) i n ^ ne right-hand side of (11. 3p . which we call the excess entropy production in 
the baths, plays an essential role in our thermodynamic approach to NESS. Moreover 
we found that the nonequilibrium entropy S((3i, (3 2 ; v) can be written as a symmetrized 
Shannon-Gibbs entropy (I2.15P in terms of the probability distribution of the NESS. 

As we have already announced in the previous section, we derive in the present 
paper an improved "nonlinear" version of the extended Clausius relation ( II. 3p . In the 
simplest case where only the parameter is changed abruptly from v to z/, and the inverse 
temperatures do not change, the new relation reads 

S(J3[, fa u') - S(fa fa v) =/?! (QT) + 02 (Qf) - ^^(W; (f3 1 Q 1 + /3 2 g 2 )> 

+ 0(e 3 5) + 0(5 2 ). (1.4) 

6 We of course have Jf s (/3i, @2] v) = — J| s (/3i, /3 2 ; v) from the energy conservation. 
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Here W is the total work done by the agent during the operation, and (A; Bj = (ABj — 
(A) stands for the truncated correlation. 

Implications of these results on thermodynamics of NESS are discussed carefully in 
section 12.31 



2 Main results 

2.1 Setup and basic definitions 

System and basic parameters: For simplicity, we treat the same class of models as 
in [5] , namely, a heat conducting system with controllable parameters (such as the vol- 
ume). More precisely we consider a classical system with iV particles whose coordinates 
are collectively denoted as 

r= (pi,...,p Ar ;r 1 ,...,rjv), (2.1) 

where pj and Vj are the momentum and the position, respectively, of the j-th particle. 
The time-reversal of a state T is 

T* = (-pi, . . . , -p N ; n, . . . , t n ). (2.2) 

The Hamiltonian H V {T) has a controllable parameter v, and satisfies the time reversal 
symmetry H V (T) = H V {T*). The system is attached to n distinct heat baths with 
(generally different) inverse temperatures (3i, . . . , (3 n . We denote the parameters of the 
model collectively as 

a =(&,..., n ;v). (2.3) 

The time evolution of the system is basically governed by the deterministic New- 
tonian dynamics corresponding to the Hamiltonian H V {T), but the effects of the heat 
baths are taken into account by a suitable Markovian dynamic^. Our results do not 
depend on details how one implements the heat baths. 

We assume that the system reaches a unique NESS after developing for a sufficiently 
long time under a given set of parameters a. By Pa(r) we denote the probability 
distribution in the NESS. 

We introduce the dimensionless measure of the "order of nonequilibrium" corre- 
sponding to a = (Pi, . . . , /3 n ; v) as 

e:= max (2.4) 

ke{l,...,n} (3 

where the "reference inverse temperature" (3 is a quantity close to Pi, ... , (3 n , which can 
be chosen rather arbitrarily. For concreteness we here set 



/3:=-f>. (2.5) 



n 

k=l 



7 The standard choice is the Langevin dynamics or the thermal wall [8]. It is essential that the 
dynamics is Markovian and satisfies the local detailed balance condition [5]. 
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Operation and path average: It is crucial for us to consider an operation by an 
external agent. During the time interval [i;, if], the agent controls the model parameters 
according to a pre-fixed protocol represented as a function 

a = (a(t)) te[t . M . (2.6) 

We write the initial and the final values as a; = ct(ii) and aif = a (if). We also denote 
by T = (r(i)) tg [ t . )tf ] a possible path of the system in the same time interval [ii, if]. We 
denote by W&(f) the stochastic "weight" for a path f , which is properly normalized so 
that 

J VT5(r(t i )-r i )Ws l (f) = l (2.7) 

holds for any initial state Tj, where the "integral" is over all the possible pathqj f . 
For an arbitrary function /(f) of a path, we define its average in the protocol a aJ^ 

(f) & := J VtftiTfa)) W fi (f)/(f). (2.8) 

Note that we have summed over all the paths, assuming that the system in the NESS 
with aj at time i = i ; . 



Entropy production and excess entropy production: Let J^(f;i) be the heat 
flux that flows into the system from the k-th bath at time i in a path T. The most 
important quantity in the present theory is the total entropy production in the baths 
defined as 

e a (r):=-/ dtJ2^(t)J k (T;t). (2.9) 
Jt > k=i 

We also introduce the excess entropy production in the baths 

rtt n 

e?(f):=-/ dtJ2Pk(t){Jk(ht)- JttHt))}, (2.10) 

Jt * fe=l 

where J| s (a) denotes the steady heat current in NESS with fixed a. Note that, for 
the stepwise operation considered in Section 11.21 ©& x (r) coincides with the quantity 
QT + 02 QT) i n (H-3j) . One can also say that the excess entropy production (I2.10p 
is a "renormalized" version of the "bare" entropy production (12. 9p which grows almost 
linearly in if — if. 

8 It is known that such a "path integral" is ill-defined (or requires an extra care in definition) for 
certain continuous-time stochastic processes. In such a case we first consider a discrete-time approxi- 
mation of the process, where the corresponding path integral is well-defined. After deriving the desired 
relations, we can take the continuum limit to recover the original process. In what follows, we shall not 
make this procedure explicit. 

9 When we define A in terms of B, we write A := B or B =: A. 
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2.2 Clausius relation and its extensions 



Original Clausius relation: Before dealing with NESS, we consider the correspond- 
ing equilibrium problem where all the baths have the same temperature. Let a describe 
an arbitrary quasi-static equilibrium operation, i.e., a(t) is an arbitrary slowly varying 
function with j3\{t) = ■■■ = (3 n (t). Then the classical Clausius relation in its most 
general form reads 

5 , eq (« f )-^ eq (a i ) = -(e A ) A , (2.11) 

which is the integrated version of (11. ip . The equilibrium entropy S eq (a) has a well-known 
microscopic expression 

S eq (a) = - j dTp^iT) log^CT). (2.12) 

Here 

(r) = ~zjW (2 ' 13) 

is the probability distribution in the equilibrium state with a, where the partition func- 
tion Z v {pi) is determined by the normalization condition j dT p e ^{T) = 1. 

Our goal is to find natural extensions to NESS of the Clausius relation (12. lip and 
the corresponding expression of the entropy like the Gibbs-Shannon expression (12.121) . 



Extended Clausius relation for NESS: As we noted in section [L~2l a key for the 

extension is to replace a diverging quantity with a "renormalized" version. In [5] we 
have shown for an arbitrary quasi-static protocol a that 

S sym (a f ) - S sym ( ai ) = -(ef) & + 0(e 2 6), (2.14) 

where the entropy production (12.91) in the original relation (12.111) is replaced with the 
excess entropy production (12.101) . Here e is the "order of nonequilibrium" defined as 
in (12. 4p . and 5 is the dimensionless measure of the change in the parameters^! a. The 
nonequilibrium entropy S sym (a) is written in terms of the probability distribution p„(r) 
of the NESS as 

s S ym(a) = -\J rfr^(r){iog^ s (r) + iog P 8 Q 8 (r)}. (2.15) 

The expression (I2.15P is similar to the Gibbs-Shannon entropy (12.121) . but has an extra 
symmetrization with respect to the time reversal. Note that for any probability distribu- 
tion which is invariant under the time-reversal, our symmetrized entropy (I2.15P coincides 
with the Gibbs-Shannon entropy (I2.12p . In particular we have S eq (a) = S sym (a) for an 
equilibrium a. 

10 If a(t) varies monotonically, one can simply take the difference ctf — cti (and then properly normalize 
each component to make the result dimensionless). For a more general function a(i), one has to consider 
the "accumulated change" such as Av := f t * dt \v(t)\. 
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The relation (I2.14p was first derived by Ruelle [7] for a class of systems with ther- 
mostat, but the expression (12.151) first appeared in [5]. 

Let us investigate the extended Clausius relation (I2.14p from the view point of opera- 
tional thermodynamics, where one examines thermodynamic quantities through various 
processes which are experimentally realizable (in principle). We then find that, by using 
the relation (I2.14p . one can determine the nonequilibrium entropy S sym (a) with the pre- 
cision of 0(e 2 ) from a thermodynamic measurement as follows. For a given nonequilib- 
rium a = . . . , (3 n ; v\ one defines its equilibrium counterpart by a eq := (/?,-■■ , P\ v) 
where /3 is the reference inverse temperature defined as (I2.5p . One then takes a as a 
quasi-static protocol which follows the straight patlJ3 from a e q to a. Then the extended 
Clausius relation (12.141) implies 

S sym (a) = S cq (a cq ) - (Qf) & + 0(e 3 ), (2.16) 

because one has 5 = 0(e) in this protocol. This expression was used in [10] for a 
numerical evaluation of the entropy in a NESS. 

The fact that we can determine S sym (a) to the precision of 0(e 2 ) might suggest 
that we are entering the regime of "nonlinear nonequilibrium." The truth, however, is 
that the extended Clausius relation (12.141) itself can be fully understood in terms of the 
physics of linear response. In fact the derivation of (12.141) in [5J was rather involved, 
and we did not realize its direct relation to the linear response theory. In the present 
paper, we shall present a much clearer derivation of (12.141) . which is essentially based 
on the linear response form (13.251) of the probability distribution in NESS. 

Remark: For the above mentioned reason, we have made in [5] a premature conclusion 
that the extended Clausius relation (12.141) goes beyond the linear response theory. We 
also remark here that the extended Gibbs relation mentioned in [5] should be reexamined 
carefully. 



New "nonlinear nonequilibrium" Clausius relation for NESS: The extended 
Clausius relation (I2.14p has an error term 0(e 2 S) which is absent in the original Clausius 
relation (I2.1ip . Although one might hope that the error may be reduced by a better 
derivation, we know from examples that there indeed exists a nonvanishing error of this 
order. This is an unfortunate fact since (I2.14p seems to be the most natural nonequi- 
librium extension of the Clausius relation f|2.11l) . All that we can hope is to derive 
improved relations with smaller error term^fi 

The main results of the present paper are "nonlinear nonequilibrium" improvements 
(I2.17p . (12.201) of the extended Clausius relation (I2.14p . The improved relations contain 
an extra nonlinear term which involves the correlation between heat and energy. While 
the extended Clausius relation (I2.14p is based on the linear response formula (I3.25p . the 

11 More precisely, we set (3j(t) = ft + {(i - (3)(t - U)/(tf - U) and v(t) = v for t E [Mf]- 

12 One may also think about a better method of "renormalizing" the divergent entropy production. 
It seems that the present prescription is the most natural one, but the naturalness may not guarantee 
that it is the "right" choice. 
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improved relations (I2.17p . ( I2.20p are derived from the nonlinear representation (" 13 . 1 8[) of 
the probability distribution of NESS which was derived by two of us (T.S.K. and N.N.) 
in [nj. See also [12]. 

Now let us describe the new relation. To characterize the amount of change in the 
protocol d, we introduce two dimensionless quantities 5 Q and 5 T . Here 5 Q denotes the 
sum of the accumulated changes (see footnote [TO]) in the parameters u(t) and j3(t) : = 
77,-1 Sfc=i flk(f)i both suitably normalized to be dimensionless. On the other hand S T 
denotes the amount of change in the degree of nonequilibrium. More precisely it is the 
sum of the accumulated changes in the relative inverse temperatures \fik(t) — (3(t)\ for 
k — 1, . . . , n. 

For an arbitrary quasi-static protocol d, we shall derive 

S sym (a { ) - S sym (a{) = -(Off + ~(W & ; Q & ) a + 0(e 2 5 r ) + 0(e 3 6 o ), (2.17) 



which is an improvement! 13 ! of fj2. 14j) since one normally haa 14 l 5 r < 0(e). Here (A; B) = 
(AB) — (A)(B) denotes the truncated correlation, and 

W & := [ l dt±-{(5(s)H u{s) (t)} (2.18) 



s=t 



is a quantity which is related to (but different from) the work. See (|2.21|) . Here /3(t) is 
the reference inverse temperature (I2.5P at time t, and the "order of nonequilibrium" is 
defined as 

£:= max wt) -m (219) 

ke{i,...,n} p(t) v ' 

te[h,tt] 

according to (12.4ft . 

We remark here that the correlation term (W&; Qa) a in (12. 171) is an intrinsically 
nonequilibrium contribution. Although both (Wa) a and (Qa) a are nonvanishing in a 
general equilibrium process, there truncated correlation is exactly vanishing. See (13. 71 p . 
We expect on the other hand that the correlation is nonvanishing whenever there is a 
nonequilibrium correction to the probability distribution of NESS. 

For a restricted class of quasi-static protocols where f}\, . . . , (3 n are fixed constants 
and only v varies, the relation (I2.17P becomes 

S; ym (a f ) - S sym (« ; ) = -(Off + — (W&; & ) & + 0(e 3 5 o ). (2.20) 
Here the quantity 

^(f) := jT *±{H«M}\^ = f\t m (2.21) 

is nothing but the work done by the external agent who operates the parameter v. 

13 We can also show that (W&; Q&) & = 0(e 2 S o ). 

14 One can indeed make S T large by, for example, letting (3k(t) oscillate rapidly in time. We here 
assume that everything is changed smoothly. 
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Figure 4: The paths in the parameter space discussed in the text, where /3& 
is the representative of /3i, . . . , /3 n . Suppose one wants to determine the entropy 
difference S sym (a') — S'syn^a) with the precision of 0(e 2 ), by using thermodynamic 
measurements. One possibility is to go thorough the indirect paths I, II, II, and use 
the extended Clausius relation (|2.14p as well as the traditional Clausius relation 
(|2.1ip . When one goes through the direct path IV, however, one has to invoke the 
new "nonlinear nonequilibrium" extended Clausius relation (|2.20p . 

2.3 Unavoidable "twist" in thermodynamics of NESS 

It may not be desirable, at least from a practical point of view, that a thermodynamic 
relation contains a correlation like (Wa", ®a) a - But we shall now argue that this term is 
unavoidable, and clearly indicates the existence of a delicate "twist" in thermodynamics 
for NESS. 

Let us consider the following example. Take two parameter values z/, v' where v — v' 
may not be small, and let 

a = i/), a' = (fix, . . . , /3 n ; v') 

a eq = (/?,... ,#1/), a' eq =(&..., (2.22) 

with (3 as in (12. 5p . See Fig. HI Suppose that one wishes to determine the entropy differ- 
ence S sym (a') — S sym (a) to the precision of 0(e 2 ) by using thermodynamic measurements. 
An indirect but clever way is to write 

S SJ m{<y') - S sym (a) ={5' sym (a / ) - 5 , sym (ag q )} 

+ {Ssym^q) - 5 , sym (a oq )} + {6^(0^) - ^^(ft)} (2.23) 

and determine the three differences separately (paths III , II and I in Fig. H]). To deter- 
mine S sym (a cq ) — S sym (a) and S sym (a') — S sym (a' eq ) , it suffices to use the (older) extended 
Clausius relation (12.141) . whose error term is 0(e 3 ) since one here has 5 = 0(e). To deter- 
mine S sym (a' eq ) — S sym (a cq ) = S'eq(aeq) — S eq (a eq ), one simply uses the standard Clausius 
relation (12.111) . which is error free. Note that all the three determinations involve only 
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direct measurements of heat currents and make use of the standard Clausius relation 
(12. lip or its natural extension^! (I2.14p . It may be reasonable to conclude that the en- 
tropy difference S sym (a') — S syra (a) (at least to this precision) is a thermodynamically 
natural quantity to look at. 

One may also determine S sym (a') — S sym (a) by using the direct path from a to a' in 
which /3i, . . . , P n are all fixed and v is varied from v to v' (path IV in Fig. H]). Note that 
the (older) extended Clausius relation f)2.14p contains the error of 0(e 2 ) since S is not 
small. To achieve the same precision of 0(e 2 ) in this case, one must use the new relation 
( I2.17P (or f)2.20p in this particular case) which contains the correlation between work and 
heat. It should be stressed that the left-hand side of (I2.20p is known (from the above 
consideration) to be a natural quantity to look at, but one must include the "nonlinear" 
correlation term in order to recover the same precision attained in the measurements 
along the three indirect paths I, II, and III. 

Obviously similar consideration applies to much more general situations, where one 
uses the most general "nonlinear nonequilibrium" Clausius relation (I2.17p . For example 
think about determining, again with the precision of 0(e 2 ), the entropy difference be- 
tween two NESS with a = (J3 U n ] v) and a" = (f3 1 + A(5, (3 2 + A(3, (3 n + A(3; v) 
where A/3 may not be small. When one considers three paths which go through equi- 
librium states it suffices to use the extended Clausius relation (I2.14p and the standard 
Clausius relation (12. lip . If one considers the direct path, on the other hand, one must 
use the new Clausius relation (I2.17p . 

The above observations, first of all, show that the correlation term in (I2.17P is manda- 
tory, and (I2.17P is essentially the unique reasonable "nonlinear nonequilibrium" exten- 
sion of the Clausius relation (12. lip . It also suggests that there is a delicate "twist" in 
thermodynamics of NESS. More precisely, we have found that one may use only the 
simple extended Clausius relation (I2.14p or must use the complicated nonlinear version 
(I2.17P ( or (I2.20p ), depending on the paths from a to a'. As far as we know, such an inter- 
esting (and a little annoying) "twist" has never been encountered in equilibrium physics. 
It may be regarded as a sharp characterization of the difficulty of thermodynamics for 
NESS. 

3 Derivation 

In what follows we derive the extended Clausius relation (I2.14p and its improvement 
(I2.17p . Although (I2.14p was derived in [5], we here present a new derivation which sheds 
better light on the nature of the relation. 

Since the derivation is somewhat involved, let us give some outline. We follow the 
standard procedure to approximate a continuous protocol a = (a(t)) 4g [ 0i t f ] by a piecewise 
constant protocol which exhibits discontinuous jumps separated by a time interval of 
It, where r is a fixed time scale which is much larger than the relaxation time. This 
procedure is carefully explained in section 13.1.31 

15 One can of course use the extended relations (I2.17[) . which is stronger, to get basically the same 
result. 
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When deriving the extended Clausius relations, we focus on the step protocol ( 13 .3|) . 
which exhibits a single jump within the time interval [— t, r]. This corresponds to a part 
of the piecewise constant protocol mentioned above. One of the keys to the derivation 
is the expression (13.41) of the excess entropy production in this step protocol. The 
expression (13 .4p only involves the difference of the probability distributions of NESS 
with different parameters, and the entropy production (0(a / ))rv^ S s * n a P a ^h starting 
from T. 

The most important ingredient in our derivation is the representation (I3.18p . which 
expresses the probability distribution p„ (r) of NESS in terms of the entropy produc- 
tion. By combining this representation (or the similar representation (I3.25P in the linear 
response regime) with the above mentioned expression (13.61) of the excess entropy pro- 
duction as well as the definition fl 2 . 1 5 [) of our symmetrized entropy, we can derive the 
step-protocol-versions (13 .7p and (I3.9P of the extended Clausius relation and its nonlin- 
ear nonequilibrium version, respectively. To be slightly more precise, when deriving the 
nonlinear relation (I2.17p . we have to carefully distinguish step protocols in which only 
the relative inverse temperatures vary and those in which only the overall parameters 
(i.e., the reference inverse temperature and the parameter v) vary. This is again carefully 
explained in section 13.1.31 



3.1 Ingredients for the derivation 
3.1.1 Some basic definitions 

Throughout the present derivation, we fix a time r > which is much larger than the 
relaxation time of the system. We denote by (a) the special protocol in the time interval 
[0, t] in which the parameters are fixed at a constant a = . . . , (3 n ; v). 

As in [HI d2], we define partially constrained path averages of any function f{T) 
of the path T as 

(/>&.:= / OT5(r(o)-r)w (Q) (f)/(f) (3.i) 

and 

(f)i:lr ■= Kinr 1 ! vtp^rmw^smr) -r)/(f). (3.2) 

As the subscripts indicate, they represent averages over histories in which the initial 
state and the final state, respectively, are specified to be T. 

In the following derivations we consider a step protocol a s in the time interval [— r, r] 
defined by 

()= fa = (&,...,&; te[-r,o) 

By S, we denote the dimensionless measure of the magnitude of the difference a' — a. 
This is a special case of the 5 introduced in the extended Clausius relation (I2.14p . 
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3.1.2 Representation of the excess entropy production 

A crucial observation for the derivations is that the excess entropy production in the 
protocol f)3.3p has an explicit representation 

mf s = JdT {pF(T) - p%(T)} (e (a0 )K ss , (3.4) 

where (a') stands for the constant protocol in the time interval [0,r]. To see this one 
notes that J dT p s ^(T) (0( a /))pl^ ss is the total entropy production in t G [0,r] (after the 

sudden change of the parameter from a to a'), and J dT p s *,(T) (@(a'))r°^ ss is the total 
entropy production in the steady state with a'. The Markovian nature of the dynamics 
is also essential here. 

Note that the left-hand side of ( 13. 4ft is the quantity for the protocol a s defined in 
the time interval [— r, r] while the right-hand side involves the quantity for the protocol 
(a') in [0, t}. The relation ( 13. 4 p is nevertheless valid since the excess entropy production 
in the time interval [— r, 0] is vanishing in the protocol d s . 

It is convenient to define 

4>(r) = p5(r)-p£(r), (3.5) 

which satisfies Ap(T) = 0(5) and J dT Ap(T) = 0. Then (13.41) is written as 

<©£>*■ = - J dT ApiT) (6 (q0 )K ss . (3.6) 

3.1.3 Prom step protocol to a quasi-static protocol 

In the following derivations, we do not derive the extended Clausius relation ( I2.14p or 
its nonlinear improvement (I2.17P directly, but prove them only for the step protocol 
(13.31) . The extended Clausius relations (12.141) for the step protocol that we will derive 
in section 13.31 is 

S sym (a') - S sym (a) = -(Off* + 0(t 2 5) + 0(S 2 ). (3.7) 

To discuss the step protocol version of the new relation (I2.17p . we first have to 
restrict the way the parameters are varied. We consider the step protocol (13. 3p with the 
restriction that 

P k = k + A/3 (3.8) 

for all k = 1, . . . ,n with a common (small) A/3. We also choose the reference inverse 
temperature as /3' = (3 + A/3. The parameter v of the Hamiltonian can vary without any 
restrictions. For the step protocol (13.31) with this restriction, we will show in section 13.41 
that 

S sym (cO - S sym (a) = -(<?>Z) &S + ~(^(/3#)(r(0)); <d & f s + 0(e 3 5) + 0(5 2 ), (3.9) 
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where A((3H)(T) is defined as 

A(PH){V) :=P'H V ,{Y)-PH V {T). (3.10) 

To get the relations (I2.14p and (I2.17P for a continuous quasi-static protocol d, we 
only need to sum up the relations (13. 7p and (13. 9p . This is a standard procedure, but let 
us explain the details since there is indeed a subtle point. 

Let us fix an arbitrary continuous protocol d re f = («ref(£))te[o,i] as our reference, and 
write a Y = a rc f(0) and af = a rc f(l). Let iV be a positive integer, and consider the time 
interval [0,tf] with U := 2Nr. For simplicity we have set the initial time as t\ = 0. We 
define a piecewise-constant protocol d on [0, tf] by 

{a t E [0, t] 
ocj t e [(2j - l)r, (2j + l)r) for j = 1, . . . , JV - 1 (3.11) 
a N t G [(2N - l)r, 2Nr] 

where the sequence a , «i, . . . , is a discrete approximation to a re {(t). For the moment 
one can naively set ctj = a ie f(j/N) (but see below). We now decompose the whole time 
interval [0, t f ] = [0, 2Nt] into N intervals [2(j - l)r, 2jr] with j = 1, . . . , N. Denote by 
atj the protocol a restricted on [2(j — l)r, 2jr]. Note that otj is a step protocol where 
the parameters jump from otj~i to aj in the middle point t = (2j — l)r. We can then 
identify each interval [2(j — l)r, 2jr] and the associated protocol dj with the interval 
[— r, r] and the step protocol d s of f )3.3p . respectively. 

Now by using the extended Clausius relation (13. 7p for the step protocol for each dj, 
we find 




N 
3=1 

= -(&T) & + 0(e 2 5) + 0{^), (3.12) 

where Sj denotes the amount of the change of parameters in aj. We noted that 0| x = 
^■^6™, and wrote 5 = J2j=i$ji anc ^ n °ted that 5j = 0(6/N). It is crucial to note 
that each (Q'^.) aj is a quantity of 0(5 j) and hence one can sum them up over the whole 
(long) time interval. By letting iV f oo, which is the quasi-static limit, we recover the 
desired extended Clausius relation (I2.14p . 

To get the nonlinear improvement (I2.17p . we need a further care. When defining the 
piecewise constant protocol (13. lip , we make small modifications so that the small step 
protocol dj satisfies the following. In the protocol dj with an even j, the parameter v do 
not vary. The inverse temperatures (3k change but with a restriction that the reference 
inverse temperature j3 defined by (12. 5 p do not change. In the protocol otj with an odd j, 
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on the other hand, all the inverse temperatures vary at once so that to satisfy (13. 8p for 
all k — 1, . . . , n (see section I3T4"|) . The parameter v may vary without any restrictions. 

Then for aj with an even j, we use the extended Clausius relation (13 .7p . and for <x,- 
with an odd j, we use the improved relation (13. 9p . Again by summing all the relations, 
we have 

N 



Ssym(oi { )-S sym (ai) = | S sym (a Tci (j^)^) - S sym (a Ic f( ^ 1 

j : even 

j: odd 
N , 



N / 
(3.13) 



where we wrote S T = $^- :even ^° = Z^:odd anc ^ ^ = ^2f=i ^j- We also noted that 

A(PH) r .= /3(2(j-l)r)H Vj (r((2j-l)T))-f3(2{j + l)r)H Vj _^ (3.14) 

vanishes for an even j, where f3(t) denotes the reference inverse temperature at time t. 
Since r is chosen to be much larger than the relaxation time, the correlation between 
A(f3H)j and 0^ can be safely neglected if j ^ j'. We can then sum up the result to get 

1 N a A 2 

S sym (a f ) - 5 sym (a ; ) = -(Off + 2 (E 4 Wi Q &Y + °( e ^) + 0(e%) + o(-). 

3 = 1 

(3.15) 



Since these quantities vary every 4r in the discrete approximation, we can write 

=(23-1)7 



^(i,), (3.16) 



where /3(s) and i/(s) in the right-hand side denote (with a slight abuse of notation) 
the smooth approximations of the reference inverse temperature and the parameter, 
respectively. Since this means that 

J2A((3H)j = J o rft-{/3( S )^ (s) }[ =4 + 0(-). (3.17) 

we get the desired (12.171) by letting N | oo in (13.151) . 
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3.1.4 Representation of the probability distribution of NESS 

The most important ingredient of our derivation is the following representation for the 
probability distribution of NESS derived by two of us in [11] 

iogp s Q s (r) = -s(a) + i{<e (a) >w r _ (e (a) )^ ss } + o( e 3 ), (3.is) 

where (a) is the constant protocol in [0, r] defined in section I3.1.1[ and S(a) is a nor- 
malization constant. We can indeed show that S(a) = S sym (a) + 0(e 3 ), but this fact is 
not used in the present derivation. See [13] . 

Let us use the reference temperature f3 of (12. 5p . and define the "nonequilibrium part" 
of the entropy production as 



* (a) (f):=- / dtJ2(P k -P)J k (t;t) 
Jo k=i 



= (Q) (f) + 0{H v {T{t)) - H u (T(0))}. (3.19) 

To get the second line, we noticed that Y^l=i Jk(P\ t) is the total heat current that flows 
into the system, and used the energy conservation. With (I3.19p . the representation 
(I3.18P is rewritten as 

logpj(r) =PF(a) -PH V {T) + l{(* {a) ) { :l r - (^a))^} + 

= /3F(a) -I3H U {T) - i/) a (T) + 0(e 3 ), (3.20) 

where we defined another constant (free energy) F(a) := J dT p^(T) H U (T) — S(a)/{3. 
We have introduced a new quantity 

:= ±{<¥ (a) >K^, - (*(«,)£!*.}, (3.21) 

which represents the nonequilibrium correction to the canonical distribution. Note that 
\&( Q )(r) is typically a quantity of 0(e), and so is ip a (T). 

Now for a general function /(f) of the path f , a naive perturbation around the 
equilibrium implies 

(/)^r=(/fc ) r + 0(/)0(e), (3.22) 

where for a = (/?i, . . . , (3 n ; v) we set a eq = (/?,••■, P; v) with (3 of ( 12. 5ft . By setting 
/ = f ( Q ), we have 

(*(a)>£ir = <« r («)>5^r + 0(£ a ), (3.23) 
because 0(^>< a ^) = 0(e). It is crucial to note that the function in the expectation value 
in the right-hand side is ^(^(r), rather than ^ r ( Qcq )(r). 
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Since the equilibrium path average has an exact time- reversal symmetr}0 (^(a))^^ = 
we further have (* (a ))Si r = + C(e 2 ) = + 

0(e 2 ), where we used the relation for (^/( a ))p"^ ss corresponding to (I3.23p . We have thus 
shown the symmetry 

(^( Q ))^r + (^w)^ ss = 0(e 2 ), (3.24) 

which will be useful. 

By substituting the symmetry ( 13.241) into the representation ( 13.201) . we see 

logp s Q s (r) = (3F(a) -0H v {Y) - (* (a) )^ ss + 0(e 2 ) 

= -S(a)-(Q (a) ) { r a l ss + 0^), (3.25) 

which indeed is a slightly different way of expressing the standard linear response for- 
mula. See the remark below. We shall see in section 13.31 that this representation is 
sufficient to derive the (older) extended Clausius relation ( 12.141) . 

Let us note that the terms (^(a)}^^ and (^(a))r*-> S s m P-241) and (@(a))r2_>. ss in 
( I3.25P all contain a contribution which grows proportionally with r, namely, the total 
entropy production o~(a) r. This means that the error terms 0(e 2 ) in (13.241) and (13. 25ft 
include the same r linear contribution. Fortunately this r linear contribution is not 
inherited by the error terms in the extended Clausius relation ( 12. 14|) or its improvement 
( I2.17p . This is because only V dependent quantities enter into these final relations, and 
the r linear contribution, which is a(a) r, just drops out. See the derivation for details. 

Remark: Let us make a few remarks about the representations of the probability distri- 
bution of NESS that we have discussed. 

As we have noted above, the term (0(a)}r*_> S s m the representation (13.251) contains 
the total entropy production a(a) r, which grows linearly in r. This is not quite desirable 
since one usually uses this kind of representation in the limit rf oo. 

One natural way to get rid of the r linear divergence is to define the excess quantity 
corresponding to (13. 19[) by 

:= " f^Y/fik - 0) {Jk(t ;t) - Jf (a)} 
Jo k=i 

n 

= * (a) (r) + r - 0) 4 s (a) = * (a) (r) + 0(e 2 ), (3.26) 

k=l 

where J^(o) = 0(e) is the steady heat current. We can then readily rewrite (I3.25P as 
logp s Q s (r) = 0F(a) -0H v {T) - + 0(e 2 ). (3.27) 



16 This is a standard result (see, for example, [12] for details). For a path f = (T(t)) te m T ] we define 
its time reversal as ft = ((T(t — i))*)te[o,Tl- Then for a function f(T) its time reversal is defined by 
/t(f) = /(ft). Then the equilibrium dynamics satisfies {f)^ T = (/^felq- Finally noting that 
J fc (ft;T-t) - -J fc (f;t), (ETSD implies (tf (a) )t = -* (o) . 
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In this new expression (^^)r*->ss 110 longer contains a term proportional to r. One can 
safely take the r f 00 limit herd 17 !. 

Another natural cure, which is suggested by (13.231) and (I3.24p . is to rewrite the 
representation (13.251) as 

logp-(r) = 0F(a) -0H„(r) - + 0(e 2 ), (3.28) 

which is a standard linear response formula for the probability distribution of NESS. 
Note that one can safely take the r | 00 limit (13. 28f) since the entropy production rate 
in the equilibrium vanishes. 

It should be stressed in passing that the higher order representations (13.181) . (13.201) 

do not suffer from the problem of r dependence. Since both (0(a))s^r and (©( a ))r*-^ss 
are proportional to a (a) r for large r, the divergence nicely cancels out. Thus one can 
also take the r t 00 limit in the representations (13.1 8ft . (13.201) . 



3.2 Derivation of the extended Clausius relation ( 13.71 ) — easier 
case 

In what follows, we abbreviate p^ s , p^ s ,, 0(a), ©(a')> ^(a); ^(a')> (' ' ')^ a \ anc l (' ' as 
p, p', O, O', ^, (•••), and (• • • )', respectively. 

As a warm up, we shall derive the extended Clausius relation ( 13. 71) in a restricted 
class of models where one has p^lT) = p s ^(T*), i.e., the probability distribution in 
NESS has a time-reversal symmetry 18 !. Although such a symmetry is in general absent, 
the derivation clarifies the relation between the linear response formula (I3.25j) and the 
extended Clausius relation (13. 7p . We consider the step protocol (13.31) where arbitrary 
change from a to a' is allowed. 

In this case the entropy (12.151) coincides with the Shannon entropy 

S Sh (a) := -J dTp s :(T) logp s Q s (r). (3.29) 
From (13.51) and a Taylor expansion, one has 
S Sh (a) = ~ f dV {p\T) - Ap(T)} \og{p'(T) - Ap(T)} 

= -j dVp\V) logp'(r) + j dYApiY) \og P '(T)- JdTp'(T)^l + 0(5 2 ) 
= S Sh (a') + J dY Ap(T) logp'(r) + 0(6 2 ) (3.30) 

17 Essentially the same consideration applies to the symmetry relation (|3.24p since the left-hand side of 
(pQ31) has a contribution proportional to r. By using (pHo) . (pQ3|) implies (*(*))^. r + (*(a))r"^ ss = 
0(e 2 ). In this form, the left-hand side does not contain any term which is proportional to r. 

18 Over damped models without momenta posses this symmetry. 
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Thus by using the assumption p'(T) = p'{T*) we see that 



Ssh(a') - S Sh (at) 



-J dTAp(T) \ogp'(r) + 0(5 2 ) 

= jdT Ap(T) (e K) )K ss + 0(e 2 6) + 0{5% (3.31) 

where we used ( 13. 25ft and noted that J dT Ap(T) const. = 0. As we have explained 
just below (I3.25p . all terms which are independent of T (including the total entropy 
production a(a) r) disappears in this process. From the representation (13.61) we get the 
desired extended Clausius relation 

S Sh {a') - S Sh (a) = -<e£>*- + 0(e 2 5) + 0{5 2 ) (3.32) 

for the step protocol of (13.31) . 

Let us also note that the above manipulation in general case (without the assumption 
Pa(T) = p^ s (r*)) leads us to the expression 

Ssh(a') - ^sh(a) = JdT Ap{V) (6 (q0 )^ ss + 0(e 2 5) + 0(5 2 ). (3.33) 

The quantity in the right-hand side can be measured numerically (but not experimen- 
tally). This expression was used in [14J for a numerical evaluation of the Shannon 
entropy in a NESS. 



3.3 Derivation of the extended Clausius relation ( 13.71 ) 



Now let us turn to the derivation of the extended Clausius relation (13.71) in the general 
case. Here we shall make use of the property 

p(T)-p(T*) = 0(e), (3.34) 

which follows from (13.251) . We again consider the step protocol (13. 3p where arbitrary 
change from a to a' is allowed. 

From the definition (I2.15P of our symmetrized entropy, we find 

S'sym(a') - S 8ym (a) 

= -\J dT{p(T) + Ap(T)}{\ogp'(T) + log p* (T*)} + ± J dT p(T) {log p(T) + log p(T*)} 

= - J dYApiv) io gP '(r*) + i J rfrz\p(r){iogp'(r*)-io g p'(r)} 
- l -f drp(r){io g p'(r)-iogp(r)}-l f drp(r){i og p'(r*) -io g p(r*)} 

=: A 1 + A 2 + A 3 + A 4 , (3.35) 
where the definitions of Ai, A 2 , A 3 , and A 4 should be evident. 
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Let us evaluate these terms separately. As for Ai, we simply repeat the manipulation 
in flCTj) . to conclude 

A 1 = -(Qff° + 0(e 2 5). (3.36) 
As for A 2 , we make use of the expansion in p(T) — p(T*) = 0(e) as 

logp(H = log[p(r) + {p(T*) - p(T)}] = logp(r) + P(r ^ ( ~ } P(r) + 0(e 2 ) (3.37) 



to find that 



M = \J dTAp(T){\ogp(T*)-\ogp(T)} + 0(5 2 
= \j dTAp{T){^-l + 0{e 2 )}+0{5 2 



2 l dT M^j^f + 0(e'5) + 0(5% (3.38) 
The treatment of A% is easy. By a simple expansion, we see that 

M = -\j dYpiv) [to g {p(T) + zip(r)} - iogp(r)] 

dT Ap{T) + 0{5 2 ) = 0{5 2 ). (3.39) 

As for A4, we repeat this estimate to get 

^ = -lj dT P^{^^ + ^)} = -ll dT P(n^ + 0(6% (3.40) 

where made a change of variable T — > T*. 

By summing up the results, A 2 and A 4 (rather surprisingly) cancel out, and we get 
the desired 

S sy m(a') - S sym (a) = -(®Z) &S + °^ 5 ) + ( 3 - 41 ) 
for the step protocol of ( 13.31) . 

3.4 Derivation of the improved relation (13.91) 

Now let us turn to the derivation of the "nonlinear" relation ( 13. 91) . 

Unlike in the previous two subsections, we only consider the step protocol (13. 3p which 
satisfies the condition (13. 8p that all the inverse temperatures (3i, . . . , (3 n of the baths and 
the reference inverse temperature /3 shift by the same (small) amount. This means that 
we have 

Pk-P = P' k -P' (3-42) 
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for all k — 1, . . . , n. The parameter v of the Hamiltonian can vary without any restric- 
tions. 

When (and only when) we make this restriction, we have 

Zh/>(r) := Vv(r) - V Q (r) = 0(e5). (3.43) 
To see this one notes from the definition (13.191) and the condition (13.421) 

(*0'«-r- (^)ss^r = { ( jT dt 4(f; t))'^-^ dt J k (t- 1))^ = 0(eS) 

(3.44) 

because — (3 = 0(e) and (• • • )' SS ^ T — (•••) S s->r = 0(5). With the corresponding 
estimate for (* / )r*-w B - (*>r*->« and the definition (EOT]) , we get0 (EQgj) . 
We also use the quantity 

VP) ■= = logp'(r) - log p(T) + 0(5 2 ) 

pin 

= {(3'F(a>) - P'H U/ (T) - W(T)} - {(3F(a) - (3 H V (T) - ^(r)} + 0(e 3 5) + 0(5 2 ) 
=: A(/3F) - A(f3H)(T) - Ai>{T) + 0(e 3 5) + 0(5 2 ), (3.45) 



where we used ( I3.20p noting thato the difference of the terms of 0(e 3 ) is 0(e 3 <5) for the 
present protocol with the restriction ( 13.421) . The final line defines A(/3F) and A(/3H)(T) 
(see also (l51Djl ). 
Note that 

r/(r) - r^(r*) = -Zh/;(r) + Zi^(r*) + 0{e 3 5) + 0{5 2 ) = 0{e5) + 0{5 2 ). (3.46) 
This implies the useful fact 

Ap4T) := l -{Ap{T) - Ap(T*)} 

= i{p(r) - Kr*)}{^(r) + + ±{ P (r) + p(r)}{^(r) - v (r*)} 

= 0(e5) + 0(5 2 ), (3.47) 
where we noted that p(T) - p(r*) = 0(e), r?(r) + 7?(r*) = 0{5), and (JS3BJ. 



19 Note that ip a (T) defined by (|3.21|) contains no terms which are proportional to r, while both 
(^(a))r*->ss anc ^ (^(a))sB->r contain the total entropy production a[a)r. Thus (|3.44[) has a term of 
0(e 2 ) which is proportional to r, but the desired (I3.43|) has no such terms. 

20 To see this one expands logp(r) into a power series in n quantities /3fe — j3 {k = 1, . . . , n) where 
the coefficients are functions of f3, v, and T. Since /3j. — (3 are the same for logp(r) and logp'(r) and 
only the coefficients change by 0(8), we see that the difference of the 0(e 3 ) terms is 0(e 3 S). 
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From the definition f 1 2 . 1 5 j) of our symmetrized entropy, we observe that 



*Ssym(o^ ) *S'sym(^) 



= - l -J dT {p(r) + Ap(T)}{\ogp'(T) + log p'(r*)} + i J dT p(T) {log p(V) + log p(P)} 

= -l| rfrzip(r){iogp'(r) + iogp'(r)}- 1 dr>(r){io g p'(r) -iogp(r)} 
+ iy rfrp(r){iogp'(r)-io g p'(r)}-i | dr P (r){io g p(r)-iogp(r*)} 

=: B 1 + 5 2 + B 3 + S 4 , (3.48) 

where the definitions of B 2 , B3, and -B4 should be evident. B2 is the same as A3 in 
f l3~35|) and hence B 2 = 0(5 2 ) from fl3T39|) . We shall evaluate 5i and 5 3 + 5 4 . 

We shall now substitute the higher order representation ( 13.1 8p of p„(r) into the 
definition of B\. Noting that f dT Z\p(r){const.} = 0, we organize Bi as 

Bi = -\ J dT Ap(T) {(9'}^r - + (&)l+r- ~ <©%-.} + 0{eH) 

= jdT Ap(T) (6%_ -\jdT Ap{T) {(e')^r + Wv-*} 

+ \j dT ApiT) {(6')^r + - (©')ss^ - + 0(e 3 <5) 

=:C 1 + C 2 + C 3 + 0(e 3 5). (3.49) 
Note that C\ is indeed what we want because (13. 6p reads 

Ci = -(®Z) &B - ( 3 - 5 °) 

Let us rewrite C 2 as 



c * = -\j dTA PP) {(e)ss^r + (e')^ ss }+o(5 2 ) = -\ 



2 [v(ry,G(T)] p + 0(6 2 ), (3.51) 

where we used (O')ss^r = (@)ss^r + 0(5), and defined 

G(r) := (6) ss ^ r + (0')^ ss (3.52) 
We also introduced the average 

[A(T)] p := J dTp(T)A(T) (3.53) 

and the corresponding truncated average 

[A(T); B(T)} P := [A(T) B(T)} P - [A(T)] P [B(T)] P (3.54) 
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for any functions A(T) and B(T) of T. In (13.511) . one first gets [rj(T) G{T)] p , but this 
can be replaced by the truncated average since [?y(r)] p = J dT Ap(T) = 0. 

To deal with C 3 , we note that {(6')^r + " (©OUr* - (©Or-*} changes 

the sign when we replace T with T*. We therefore recall (I3.47P and write 

C, = \JdT Ap_(T) {(6'>^r + (©')r^ss - ~ 

= \jdT Ap_(T) {<*'>^r + - WLt- " <*%-*.}, (3-55) 

where we used (13.191) . Now recalling the approximate time- reversal symmetry (I3.24p . 
one finds^] {■ • • } = 0(e 2 ). Combining this estimate with (I3.47p . we see that 



2r2\ 



C 3 = 0{e 6 5) + 0(e 2 S 



(3.56) 



It now remains to evaluate B$ + B±. By making a change of variable r — > T* in B4, 
we can rewrite B% + B^ as 

B, + B A = - l -j dT{p{T*)-p{T)}{\ogp'{T)-\ogp{T)} 

= -\JdT {p(T*) - P (T)}{A(PF) - A(J3H)(T) - A^(T)} + 0(e*5) + 0(e5 2 ) 

where we used (13. 45ft . Noting that A(f3F) — A((3H)(T) is invariant under r — > T*, we 

get 



i J dT {p(n - p(T)} A^(T) + 0(e 4 5) + 0(e5 2 



(PJP 



l);A^{T) 



J p 



2 [\ p(T) 

Here we used the truncated average since 

p(r*) 

.P(T) 



+ 0(e 4 5) + 0(e<5 2 ). 



dT {p(T*) - p(T)} = 0. 



By using (I3.20p . and recalling that .ff„(r*) = i7„(r), we see 
p(T* 



P(T) 



1 = exp [-V>(r*) + V(r) + 0(e 3 )] - 1 = i/j(T) - ^(r*) + 0(e 2 



(3.57) 



(3.58) 



(3.59) 



because ^(r) = 0(e). We substitute this into (I3.57P and recall that Aip(T) = 0{e8) as 
in QS3SD to get 



B 3 + B 4 = - [A^(T); {m - i>(T*)}] p + 0(e 3 5) + 0(e5 2 



1 

4L 



^(r);{<*) 



r*->-ss 



j p 

(^)ss^r 



r— >ss 



(*) ss ^r*} +0(e 3 5) + 0(e5 2 



Note that r linear contributions completely cancel out in the sum. 
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where we used the definition (13.211) of Again from the approximate time-reversal 

symmetry (13 . 24[) and Aip(T) = 0(e5), we have 

1 r ZhHr); { Wr-> ss + (*> ss ->r}l + 0(e 3 S) + 0{t5 2 



p 



A^{T); {(e> r - aa + (6) ss ^ r } + 0(e 3 <5) + 0(e5 

J p 



2 [^V(r); G(T)] P + 0(e 3 5) + 0(5 2 ), (3.60) 

where we again noted that (©')ss->r = (@)ss^r + 0(5), and used the definition (I3.52p . 
Note that the T independent r linear contribution again plays no roles since [/(r); const.] - 
for any function f(T). 

Now by summing (13301 . fl33T|) . fl336l) . and (13301 . we get 

S sym (a') - S sym (a) = -(QZr ~ \ \W) + ^(F)}; G(T)] p + 0(e 3 5) + 0(5 2 ). (3.61) 
Since (I3.45P implies 

rj(V) + A^{V) = A(/3F) - A(f3H)(T) + 0(e 3 5) + 0(<5 2 ), (3.62) 
(13 .611) is rewritten as 

S sym (a') ~ S sym (a) = -(<3>T E f s + \ [A(PH)(T); G(T)] p + 0(e 3 5) + 0(5 2 ), (3.63) 

where A(f3H){T) is defined in flSTTUj) . 

It now remains to interpret the truncated correlation in ( I3.63[) . We claim that, for 
any function A(T) of T, one can rewrite the correlation function as 

[A(T) G(T)] P = (A(T(0)) Q &s r (3.64) 

where the right-hand side is the path average in the step protocol (13. 3p . By using this, 
(I3.63P becomes 

S sym (cO - S sym (a) = -{QTf* + l(A(PH)(T(0)); Q & f s + 0(e 3 5) + 0(5 2 ), (3.65) 

which is our goal (13 .9p in the present section. 

To show the claim (I3.64p . we substitute the definitions (13.521) and (I3.53P to get 

[A(T) G(T)] p = J dTp^T) A(T) (6 (o) )W r + J dT p™(T) A(T) (6 (a0 )K ss (3.66) 

The key observation is to consider the step protocol ( 13.3p . and decompose a path 
T = (T(t))tE[-T,T] in the interval [— r, r] into two parts T_ = (r_(t))t e [_ Ti o] and T + = 
(r + (t)) te [o, r ]. One then identifies (• • • with the path average over f _, and (• • • )p°^ ss 
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with that over f + . Then the claim f !3.64j) may be intuitively clear, but let us present a 
careful derivation for completeness. 

We first note that, since the process is Markovian, the original path average is related 
to the decomposed path averages through 

j vf '(•••) = J vr_ J vt + 5(r_(o) -r + (o))(---) (3.67) 

and 

W a .(f)=W(a)(f_)W(« (r + ). (3.68) 
By substituting the definition (13. 2p . we find 

drp-(r)A(r) (e (o) >^ r 

= JdTAiv) y , 2?f_^(r(-r))w ( a)(f , -)tf(r_(o)-r)e (a) cr_) 

= / i?f_p s Q s (r(-r))w (a) (f_)A(r_(o))e {Q) (f_) 

= j pfp-(r(-r))w &s (f)A(r(o))e {Q) (f_), (3.69) 

where we used the normalization f VT + 5(T + (0) — r)W( a ')(r + ) = 1. Similarly we 
substitute the definition (13. ip to get 



/ 



dV p s *(T) A(T) <e (Q0 >K ss 



dry a s (r)A(r) | pf + 5(r + (o)-r)w (a0 (f + )e (a0 (f + ) 
pf + p^ s (r + (o)) w (a0 (f + ) A(r + (o)) e (a0 (f + ) 
= J OT P s j(r(-r))w (is (f)A(r(o))e {QO (f + ), (3.70) 

where we used the invariance p„(r) = f UT_ p^ s (r_(— r)) W( a )(r_) 5(r_(0) — T) of 
the NESS. Adding (13T69|) and (13~70|) . noting that e« s (f) = 0( a )(f_) + 6 (a /)(f + ), and 
recalling the early definition (I2.8p . we get the desired claim (I3.64p . 

Finally let us show that, as was noted in section |2~2"| the correlation term 0q,)° 
vanishes in an arbitrary process in equilibrium. To see this first note that the quan- 
tity (0) ss _j.r + (@)r^ss exactly vanishes in an equilibrium process where ©a(r) = 
P{H v (T(0)) - H„(T(t))}. Then the definition f l332|) implies that G(T) = 0(5), and 

ll6IlC6 

[A((3H)(T);G(T)} p = 0(S 2 ), (3.71) 
which has no contribution in the quasi-static limit. 
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4 Summary and discussions 



In the present paper, we have presented a careful and detailed account of our approach 
to extend entropy and thermodynamics to nonequilibrium steady states (NESS) in a 
Markov process describing a heat conducting system. 

Extended Clausius relations and the "twist" in thermodynamics of NESS: 

Among various possible routes toward the extended entropy for NESS, we have taken 
the one which seems to be the closest to the historical introduction of the notion of 
entropy. We examined how the Clausius relation S' eq — S cq = Q/T = — (0) (see (12. lip ), 
where G is the total entropy production in the heat baths, and the corresponding Gibbs- 
Shannon expression f 1 2 . 1 2 j) can be extended to a NESS. We have then found in [5] that 
a very natural extension S' sym — S sym ~ — (6 ex ) (see (I2.14p ) was possible, where 9 ex is 
the excess (or "renormalized" ) version of entropy production in the baths. The excess 
entropy production G ex characterizes the intrinsic entropy production caused by the 
operation of the outside agent. 

The extended relation is simple and looks natural until this point. In the present 
work, however, we found that if we use the same nonequilibrium entropy S'sym and 
try to determine this quantity with the precision of 0(e 2 ), we must sometime use the 
"nonlinear nonequilibrium" improvement S' sym — S sym ~ — (6 ex ) + (W; 0)/2 (see (I2.17P ) 
of the extended Clausius relation, which includes a correlation between the work and 
the heat. Although the new relation does not quite look like a thermodynamic relation, 
we found that this nonlinear improvement is mandatory and unique once we accept the 
naturalness of the extended Clausius relation (I2.14p . Furthermore, it was found that 
when one wants to determine the difference of entropies in two different NESS with the 
precision of 0(e 2 ), one may use only the simpler extended Clausius relation without a 
nonlinear term or must use the nonlinear relation, depending on the operational paths in 
the parameter space one takes. The discovery of this annoying but unavoidable "twist" 
in thermodynamics of NESS as well as the new "nonlinear nonequilibrium" extended 
Clausius relation is the main contribution of the present paper. 

Unified derivation of the extended relations: From a technical point of view, we 
have also made a nontrivial progress by developing a new method for deriving Clausius- 
type thermodynamic relations for NESS. The new method is much more straightforward 
and transparent than the one we used in the older work [5] . The key strategy is to use the 
simple expression ( 13 .4[) for the excess entropy production along with the representation 
(I3.18P of the probability distribution of NESS. 

The new derivation made it clear that the previously derived extended Clausius 
relation (I2.14p should be understood as a result within the linear response theory (in a 
broad sense). It led us to the improved extended Clausius relation ( I2.17P which explicitly 
contains the "nonlinear nonequilibrium" term. 

It is of course straightforward to extend the present results to other classical systems 
exhibiting NESS. In particular models with particle flows can be treated in the line 
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of Section 6 of [12J. Very recently the extended Clausius relation f |2.14|) was derived 
in a quantum system by extending the present method [TS]. It was found that the 
nonequilibrium entropy is expressed as the symmetrized von Neumann entropy 



*Ssym [Ps 



2 



{\ogp ss + log(TpssT)} 



(4.1) 



where T is the density matrix for the NESS and T is the time reversal operator. We still 
do not know whether the "nonlinear nonequilibrium" relation (I2.17P can be extended to 
quantum systems or not. 



Entropy for NESS: We have also argued that the symmetrized Gibbs-Shannon en- 
tropy (12.151) is a natural entropy from the point of view of operational thermodynamics. 
More precisely if we stick on the natural prescription for "renormalizing" the divergent 
entropy production, this is the unique entropy to the second order in the "order of 
nonequilibrium" e. 

Note that our symmetrized entropy (12 . 1 5[) can be seen as the sum of the Shannon 
entropy of p„(r) and the relative entropy between p^iX) and its time-reversal p^ s (r*). 
Therefore the symmetrized entropy certainly reflects the amount of breakdown of the 
time-reversal symmetry in a NESS. Unfortunately we still do not have any further in- 
terpretation of the new entropy. It would be quite exciting if one can read off deeper 
unknown structure of statistical mechanics for NESS from this rather suggestive defini- 
tion. 

A very important issue that must be investigated is to find possible relations between 
our entropy for NESS and those derived through the studies of large deviation properties 
[21 E]. Although we do not see any direct connections for the moment, we believe that 
the thermodynamic-like construction in [3] should be investigated in the light of what 
we have found. 



Future issues: There are still many issues in thermodynamics for NESS which must 
be carefully studied. The situation about the second law, i.e., the possible inequalities 
corresponding to the equalities (12.141) or (I2.17p . is still rather complicated [13]. 

We also note that it is possible to derive a formally exact results which correspond 
to our thermodynamic relations. For example, for any process in which the initial and 
the final states have the same reference inverse temperature (I2.5p . we can show that 

1 / e -*at/ 2 \«t 
FU°0 - = p \ e -*J )& . ( 4 - 2 ) 

where a} is the time-reversal of any path a, and $q = + Here i*f u n(a;) is the 

nonequilibrium free energy (to the full order in e) and coincides with the equilibrium 
free energy in equilibrium states. The relation (14. 2 j) reduces to (symmetrized version of) 
the Jarzynski work relation [16] when the initial and the final states are in equilibrium. 
One can define the nonequilibrium entropy to full order by using the standard formula 

Sfuii(a) := P{ J dTp s :{T) H U {T) - F full («)}. (4.3) 
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It is found that this new entropy coincides with the symmetrized entropy (1 2 . 1 5 [) to the 
precision of 0(e 2 ), but is not exactly the same as (|2.15p . See [T3] for details. The 
physical meaning of the suggestive result (14 .2p is still unclear to us. 

So far all of our results are derived using rather formal perturbation in the "degree 
of nonequilibrium" e. The mathematically minded reader might ask if the results can 
be formulated as mathematically rigorous statements. If one reexamines our derivation, 
it will turn out that most of the procedures are mathematically sound in principle (if 
one has suitable nice properties about convergence etc.) and the key is whether the rep- 
resentations (I3.18p . (13.251) can be made rigorous. We believe that these representations 
can be rigorously stated with some extra work if we make suitable assumptions on the 
Markov process (see, for example, p2]). Such naive rigorous estimates, however, will 
have very small range of applicability. It is quite likely that we should take smaller and 
smaller e as the system size gets larger. There is thus no hope of controlling thermody- 
namic limit with the current technique. It seems that we need a revolutionary new idea 
to resolve this issue and obtain meaningful results for large nonequilibrium systems. 

Last but not least we wish to stress that both the extended Clausius relation (12. 14j) 
and its "nonlinear nonequilibrium" improvements (I2.17P and (I2.20p may be investigated 
experimentally in principld^E It is an exciting challenge to directly observe the "twist" 
in nonequilibrium thermodynamics in experiments. 
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